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The literature reveals that educators find it challenging to foster critical thinking (CT) in their 
students if they have not learned how to use CT in their educational system or training. This paper 
reports findings from a national research project that was undertaken to enhance the educators’ 
ability to promote CT in their teaching practices. Using a randomized control trial design with a pre- 
and post-test, 91 educators from 14 of the 17 schools of nursing in Pakistan consented to enroll in 
the study and 72 completed the study. The intervention included 40 hours of learning experience 
during two workshops that focused on CT. Data were collected, pre- and post-intervention, via 
observations and audiotaping of the participants teaching sessions for 60-90 minutes. The data 
obtained was assessed for the educators’ level of questioning, teaching strategies, and facilitation 
skills. Data were analyzed using descriptive and inferential statistics. Compared with the pre¬ 
intervention data, findings from the post-intervention data in the experimental group revealed 
positive changes in their pedagogical skills, including a significant increase in the number of higher 
order questions that are considered important for developing students’ CT skills. This study affirms 
that educators must have structured training to use and foster CT in their teaching practices. 


As critical thinking (CT) is an important attribute in 
intellectual development, knowledge acquisition, and 
knowledge utilization in individuals, teachers are expected 
to nurture students’ critical thinking skills (Kong, 2006; 
Loving & Wilson, 2000; Wangensteen, Johansson, 
Bjorkstrom, & Nordstrom, 2010). It is assumed that 
teachers know what CT is and how it can be promoted in 
their teaching practices, but this may not be true (Choy & 
Cheah, 2009; Mangena & Chabeli, 2005) unless they have 
learned it in their pre-service or in-service training (Kong, 
2006). This is particularly true for a country like Pakistan 
where the focus of education for students is predominantly 
rote learning rather than acquiring CT skills, and for 
teachers as the givers of knowledge rather than facilitators 
of students’ knowledge development (Davies & Iqbal, 
1997; Dean, 2005; Gul et al., 2010; Siddiqui, 2007). 
Teachers must emulate CT if they expect their students to 
do so. In addition to having a command over the subject 
to be taught, teachers should understand the “conceptual, 
strategic, epistemological, and educational ramifications 
of critical thinking” (Mangena & Chabeli, 2005, p. 293). 
To promote students’ CT, teachers need to select 
appropriate content and instructional strategies to address 
the learning objectives, and they should facilitate teacher- 
student interaction, encourage students to ask thought- 
provoking questions, and respond to their questions 
without bias (Ijaiya, Alabi, & Fasasi, 2010; Zygmont & 
Schaeffer, 2006). 

The literature on higher education reveals an 
increasing interest in investigating faculty understanding 
of CT (Cassum, Profetto-McGrath, Gul, Ashraf, & 
Kauser, 2013; Moore, 2011) or their dispositions towards 
CT (Choy & Cheah, 2009; Duron, Limbach, & Waugh, 
2006; Emir, 2009; Hsu, 2007; Mangena & Chabeli, 


2005; Ovais, 2007; Profetto-McGrath, Smith, Hugo, 
Patel, & Dussault, 2009; Zygmont & Schaeffer, 2006). 
Several researchers have also investigated the 
questioning skills of teaching faculty. Findings from 
most of the studies suggest that faculty members need to 
improve their pedagogical skills (Ball & Garton, 2005; 
Choy & Cheah, 2009; Mundy & Denham, 2008; Nicholl 
& Tracey, 2007; Zygmont & Schaeffer, 2006). However, 
limited research is available detailing how to improve the 
teachers’ pedagogical skills to foster the students’ CT. 

This paper reports findings from a research project 
that was undertaken in Pakistan at the national level to 
enhance the CT skills of educators in the planning and 
delivery of their curriculum. This paper focuses on the 
results pertinent to the following questions: 

1. Do educators, after attending the intervention 
as part of the experimental group, ask higher 
level questions in their classroom discourses 
than those who are in the control group? 

2. Do educators, after attending the intervention 
as part of the experimental group, use more 
active teaching methods compared to 
educators in the control group? 

3. Is there any difference in the educators’ 
facilitation skills before and after the 
intervention? 

Literature Review 

Description and Significance of Critical Thinking 

Literature is replete with various descriptions of 
CT because it can be explained from different 
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paradigms such as analytical philosophy and logic, 
scientific method (testing hypothesis), pragmatism, 
psychoanalysis, and critical theory (Brookfield, 2012). 
Moreover, CT is a multidimensional concept that can 
be viewed as a tool, set of skills, process or outcome 
(Cassum et al., 2013; Moore, 2011). However, CT is 
generally considered a subset of the reflective process 
that helps individuals make sound judgments because it 
involves thorough assessment and scrutiny of 
information before arriving at conclusions (Daly, 1998; 
Dewey, 1916). Critical thinking helps the individual to 
identify and check one’s own assumptions and those of 
others (Brookfield, 2012; Paul, 1993) and thus 
“represents a major qualification for people in deciding 
what to do or believe” (Yang & Chou, 2008, p. 683). 
Similarly, CT is useful to analyze complex data, 
evaluate situations and actions, and implement the most 
appropriate actions; hence, it is a must have skills for 
effective problem-solving and decision-making in all 
walks of life—social, clinical, ethical, managerial, or 
political (Simpson & Courtney, 2002). 

Because of its importance in knowledge 
development, assessment and utilization (Paul, 1993), 
CT is considered vital in modern education especially 
in higher education (Brookfield, 2012; Kong, 2010). 
Moreover, CT is expected to be an integral component 
of teaching pedagogies in every discipline, particularly 
the health care disciplines (Behar-Horenstein & Niu, 
2011; Cassum et al., 2013; Daly, 1998; Paul, 1993; 
Velde, Wittman, & Vos, 2006). Ethical, efficient and 
effective care requires sound clinical judgment that is 
not only grounded in thorough knowledge, but also 
requires one’s ability for critical thinking, analytical 
reasoning, decision-making, and reflective practice 
(Moeti, van Niekerk, & van Velden, 2004). 

Development and Facilitation of Critical Thinking 

Although there is no one right way to teach or assess 
critical thinking, literature suggests that teaching 
approaches requiring active students’ involvement instead 
of didactic teaching practices are critical to promote and 
facilitate CT (Simpson & Courtney, 2002; Velde et al., 
2006). Teaching approaches that focus on content instead 
of process (Sellappah, Hussey, Blackmore, & McMurray, 
1998) or, in other words, on what to think instead of how 
to think, do not facilitate CT. Teaching strategies such as 
problem-based learning, writing reflective journals, role- 
playing, concept-mapping, and debates are reported to 
help (Simpson & Courtney; Velde et al., 2006; Yang & 
Chou, 2008) because these strategies help engage students 
in their learning process and can foster their CT 
dispositions (e.g., inquisitiveness, analytical abilities, 
reasoning skills, self-confidence, and open-mindedness; 
Chan, 2012; Ennis, 1993; Paul, 1993; Vacek, 2009; Velde 
et al., 2006). 


Clasen and Bonk (1990) posited that although there 
are many strategies that can impact students’ thinking, it 
is the teachers’ questions that have the greatest impact. 
Research evidence consistently suggests a direct 
relationship between the types of questions posed by 
faculty and the students’ ability to develop CT (Redfield 
& Rousseau, 1981; Rossignol, 2000; Shim & Walczak, 
2012). Higher level cognitive questions require learners to 
manipulate information to create and support responses, 
while lower level cognitive questions are answered 
through recall, recognition, and simple application of 
information. Therefore, the former is considered 
congruent with CT (Redfield & Rousseau, 1981). Several 
descriptive studies in nursing suggest that many teachers 
use factual and lower level questioning which does not 
promote CT (Myrick & Cpsych, 2002; Nicholl & Tracey, 
2007; Phillips & Duke, 2001; Profetto-McGrath, Bulmer, 
Day, & Yonge, 2004; Sellappah et al., 1998). However, 
there is some evidence in the literature that a specific 
module pertinent to CT and questioning skills can 
enhance the educators’ ability to ask higher level 
questions (Craig & Page, 1981; Wink, 1993). 

Since CT is a social learning process, students can 
leam it from their peers and faculty modeling 
(Brookfield, 2012). However, the demonstration of CT 
necessitates intellectual discipline, self-evaluation, 
counter thinking, opposition, challenge, and support 
(Paul, 1993). Empirical evidence suggests that teacher- 
student interaction and interaction among students 
influence the students’ cognitive and affective learning 
outcomes (Dorman, 2012; Gul, Barolia, & Moez, 2013). 
A learning environment that is affirmative, constructive 
and rewarding is likely to foster thinking (Billings & 
Halstead, 2009). Developing the students’ ability to think 
critically is influenced by the teachers’ competence and 
approach to teaching (Simpson & Courtney, 2002). A 
positive gain in students’ CT is reported by Smith (1977) 
when “faculty members encouraged, praised, or used 
students’ ideas” (Shim & Walczak, 2012, p. 16). The 
educators’ own values, interest, and dispositions towards 
CT can also influence students’ thinking and learning 
(Kong, 2006; Mangena & Chabeli, 2005; Ovais, 2007; 
Profetto-McGrath et al., 2009). If teachers aim to prepare 
students at a higher level of cognitive thinking, “they 
must first emulate higher level thinking in their 
instructional practices” (Ball & Garton, 2005, p. 59). 
Likewise, Facione and Facione (1996) asserted that CT 
needs to be demonstrated and that demands constant 
metacognitive reflection on “what one is doing and why” 
(p. 133). Thus, the educators’ role modeling and 
mentoring are necessary to promote CT (Brookfield, 
2012). Explanation of abstract concepts and well 
organized presentations are found to impact students’ CT 
as well (Shim & Walczak, 2012). 

Contrary to the required teaching practices, 
didactic teaching and rote learning are still prevalent in 
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most teaching institutions and disciplines in Pakistan, 
and nursing education is no exception (Davies & Iqbal, 
1997; Kamal, 1999; Khalid & Khan, 2006). Moreover, 
considering the socio-cultural dimension of CT, 
Pakistani learners may be viewed as members of a 
culture that does not encourage questioning people 
who, by virtue of their age or position, are in authority. 
In their study on teacher education, Davies and Iqbal 
(1997) reported that the majority of teaching was 
lecture based, and notes were dictated to students. In 
certain cases, some students did not take any notes, but 
just listened to the lectures and then used the textbooks 
to prepare for the examinations. Similarly, in 1998, a 
nationwide study of nursing schools in Pakistan 
indicated that nursing students were not encouraged to 
think and question (Kamal, 1999). A comment made by 
a student reflects this state of affairs: “If I say, ‘I have 
not understood’, I am told, ‘No need to understand, just 
remember it’” (Kamal, 1999, p. 43). 

In view of the above literature, the research 
intervention was proposed to enhance the educators’ 
pedagogical skills for promoting CT in their students. 
The following assumptions were identified as part of 
the design and implementation of this study: 

• Critical thinking skills can be developed with 
practice. 

• Educators can promote students’ CT if they 
know how to promote it. 

• Students’ critical thinking can be developed if 
educators ask higher level questions, use 
active teaching strategies, and demonstrate 
good facilitation skills. 

• Educators’ attitude and knowledge of CT are 
reflected in their teaching practices. 

Methodology 

Study Design 

In pursuit of a better quality of evidence (Polit & 
Beck, 2008), we employed a randomized control trial 


design with a pre- and post-test after the intervention. 
The independent variable was teachers’ training and the 
dependent variables were their level of questions, 
teaching strategies and facilitation skills. The study was 
completed over a 2-year period (February 2009 to 
March 2011) in three phases—pre-test, intervention, 
and the post-test—as illustrated in Figure 1. 

Definition of Terms 

For the purpose of this study, the term educators 
meant teachers or faculty members regardless of their 
disciplines, but who were teaching in Bachelor of 
Science in Nursing (BSN) programs in Pakistan. Based 
on the hierarchy of Bloom’s (1956) taxonomy of 
cognitive thinking, questions requiring knowledge 
recall, comprehension, and simple application were 
considered lower level questions while questions 
requiring complex thinking (e.g., analysis, synthesis, 
and evaluation skills) were considered as higher level 
questions. Based on the work done by Van Amburgh, 
Devlin, Kirwin, and Quakers (2007), “active teaching 
methods” referred to any teaching strategy that 
involved active engagement of students for a specific 
purpose; the activity began with some instructions by 
the teacher (context, process, and timings) and ended 
with students’ reflections on the learning from the 
activity. “Facilitation skills” referred to the teachers’ 
behavior that had the potential to affect students’ 
motivation for participation in the class (Van Amburgh 
et al., 2007). 

Population and Sampling 

The study population comprised all full-time 
nursing and non-nursing faculty members who taught in 
BSN programs in Pakistan. Following a universal 
sampling technique, the 148 faculty members who were 
eligible from 17 schools of nursing in the country were 
invited to participate in the study. Part-time teachers 
were excluded from the study to avoid envisaged 
complexities with regard to seeking permission and 
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commitment from their institutions. Ninety-one 
teachers (61%) from 14 schools of nursing consented to 
participate; of these, 44 participants were randomly 
allocated to the intervention group and 47 to the control 
group. All 91 participants were available for the first 
observation. However, as illustrated in Figure 2, 19 
participants (nearly 21%) were lost from both groups 
over the course of the study while 72 participants 
completed the study and were included in the analysis. 
The attrition rate and reasons did not differ between the 
intervention and control groups. 

Recruitment of the Participants 

The list of schools offering a BSN was obtained 
from the Pakistan Nursing Council. After approval of 
the institutional review board (1064-SON-ERC-08), a 


letter of information about the study was sent to the 
head or principal of each school. A written consent and 
a list of full-time faculty members were requested if the 
Head of the schools supported their faculty 
participation in the study. An informed consent was 
obtained from each participant before the first point 
(see Figure 1) of data collection. 

Intervention 

The intervention consisted of two learning 
workshops (total 40 hours of direct contact) that were 
conducted 14 weeks apart as illustrated in Figure 1. The 
intervention was developed and implemented by the 
research team, which consisted of three educators from 
nursing, two from basic sciences, and one from English 
Language. Although not identified at the outset 


Figure 2 

Recruitment and Retention of the Study Participants 














































Gul et al. 


Promoting Critical Thinking 41 


of our study, our approach to intervention closely 
resembled the DASK (dispositions, attitudes, skills and 
knowledge) model of teaching thinking skills by Kong 
(2006, 2010). Guided by the three dimensions of critical 
thinking—knowledge, skills, and attitude (Paul, 1993; 
Rubenfeld & Scheffer, 2006; Staib, 2003)—the 
following learning outcomes were set for the 
intervention: 

• Appreciate the significance of practicing and 
promoting CT in nursing. 

• Identify skills and attitudes required of critical 
thinkers. 

• Write instructional objectives for each level of 
Bloom’s taxonomy. 

• Differentiate between active and passive 
learning strategies. 

• Identify characteristics of effective questioning 
in teaching. 

• Generate questions at each level of Bloom’s 
taxonomy. 

• Convert lower order questions into higher 
order questions. 

• Identify ways to mitigate the barriers to 
teaching critical thinking. 

• Identify ways and strategies necessary to 
promote CT in students. 

Teaching content and its delivery (Appendix A) to 
facilitate the above outcomes were selected based on a 
thorough literature review on critical thinking, the 
team members’ experience as educators and the 
pedagogical skills of teachers observed in Phase I of 
the study (Gul et al., 2010). A folder including the 
learning objectives, teaching strategies and related 
readings was given to each participant on day one of 
the workshop. The focus of the first workshop was to 
explore the educators’ understanding and attitude 
about CT, clarify misperceptions, and help them 
recognize the contextual factors that could affect one’s 
ability to think in a learning environment. Moreover, 
types, levels, and effective questioning techniques 
were addressed. The importance of questioning by 
faculty and students was emphasized. Bloom’s (1956) 
taxonomy of educational objectives was used to 
develop the participants’ skills in writing behavioral 
objectives and in asking higher order questions. In 
addition, the concept of alignment between objectives, 
teaching strategies, and assessment strategies was 
included in the first workshop. Active teaching and 
learning strategies (Van Amburgh et al., 2007; 
Rubenfeld & Scheffer, 2006) including group work, 
games, concept maps, debate, and reflections were 
used to address the selected content. 


At the end of the first workshop, the participants 
were instructed that during their regular work, they 
reflect on what they learned in the workshop and 
whether they could apply their learning in their teaching 
practice. They were asked to submit a one-page 
summary of their reflections to the primary investigator 
two weeks prior to the second workshop. Information 
obtained from the participants’ reflections, especially 
the obstacles they may have faced in promoting CT, 
were addressed in the second workshop. In addition, 
they were asked to bring a course syllabus/grid of any 
course they had recently taught or were currently 
teaching. After removal of the institutions and 
instructors’ identifying information, these course grids 
were critiqued to understand/learn curriculum 
alignment: the objectives, teaching strategies, and 
assessment strategies. At the end of the workshop, the 
participants were asked to evaluate the intervention 
workshops based on the identified learning outcomes. 
The post-intervention data were collected 12 weeks 
after the second workshop. Considering the nature of 
our research questions, we did not aim to follow 
teachers in a specific course, or for them to be with the 
same students as at the first point of data collection 
(pre-intervention), but teaching a course in the same 
program was the criteria. 

No training was offered to the control group until 
the second set of data was collected. A three day 
condensed workshop of similar content as was offered 
to the experimental group was held for the participants 
in the control group in order to provide them with 
necessary knowledge and skills pertaining to CT. 
Considering the expected number of participants (more 
than 40) in each workshop (both in the experimental 
and control groups), each workshop was offered twice. 
Based on the logistic consideration and the number of 
participants from different cities, one set of workshops 
was conducted at a nursing college in Islamabad, and 
another set of workshops was offered at a nursing 
school in Karachi. Therefore, the total number of 
participants was almost equally divided between the 
two venues. 

Data Collection 

Data were collected pre- and post-intervention 
through classroom observation of the participants’ 
teaching sessions, which lasted from 60 to 90 minutes. 
Moreover, proceedings of their classes were audiotaped 
to obtain data on the teachers’ questions. A structured 
checklist was used to record contextual information on 
the class (e.g., class size, duration, and physical 
environment), types of teaching strategies, and the 
teachers’ facilitation skills (see Appendix B). Field notes 
were recorded to substantiate the ratings on the 
structured list and anything that could have impacted the 
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students’ thinking. The field notes were helpful in 
capturing the teacher-student interaction and other 
behaviors related to questioning, for example, several 
questions were self-answered by the teachers (see Gul et 
al., 2010 for more details). Demographic information of 
each participant was obtained at the time of their consent. 

The research team developed the observation 
checklist based on the literature about teachers’ 
pedagogical skills affecting students’ thinking and class 
participation. The facilitation skills included five items: 
(a) teacher-student interaction (e.g., eye contact, 
listening), (b) attitude of mutual respect (e.g., language, 
interaction tone), (c) responsiveness to students’ 
concerns (e.g., clarifying a concept, identifying a 
resource), (d) encouragement given to students for 
asking questions (e.g., acknowledgement, appreciation), 
(e) and dictation of notes to students. The first four 
items were considered desirable for promoting students’ 
thinking and participation, while the last item was 
considered undesirable for developing students’ CT 
skills. The items were in question format and were 
measured using an ordinal scale from not at all to some 
extent to a great extent. The research team members 
established content validity of the checklist. The 
observation process was pilot tested before the actual 
data collection. Some tweaking of the checklist, 
including identification of behaviors for each item of 
facilitation skills and issues that related to clarity of the 
recordings, was done based on the pilot testing. 

Data Analysis 

The recorded participants’ data on questioning was 
transcribed verbatim and verified with the recordings 
by the research assistant. Based on Bloom’s (1956) 
taxonomy for cognitive thinking and the questioning 
framework offered by Profetto-McGrath et al. (2004), 
the teachers’ questions were coded for types and levels 
of thinking (i.e., knowledge, comprehension, 
application, analysis, synthesis, and evaluation; see 
Appendix C). Questions that lacked clarity or had 
multiple interpretations were coded as vague. 
Rhetorical questions or those that posed for probing, 
facilitation, or determining students’ reactions to a 
situation were categorized as “other types.” A question 
posed and then instantly answered by the teachers 
without giving any chance to the students to respond, or 
questions with answers written on the same slide of a 
PowerPoint presentation were coded as rhetoric 
questions. If for any reason, the teacher repeated a 
question, it was counted only once. 

After coding, data were entered into an SPSS 
database. For each participant, we entered the number 
of questions in each of the categories: high order, low 
order, other types, and vague. The coded data from the 
observation checklists were also entered. 


Descriptive statistics were computed for 
participants’ characteristics, data obtained through the 
structured checklist on classroom observation and the 
teachers’ questioning. Pearson product moment 
correlations were used to examine the relationships 
between the variables. To control for the effect of class 
duration on the number of questions, the number-of- 
questions variables were re-expressed by dividing 
questions on the duration. To test for finding 
differences between pre-and post-intervention and 
between the intervention and control group, we 
conducted a mixed between-within analysis of variance 
(ANOVA), a = 0.05. 

Results 

Characteristics of the Participants 

As shown in Table 1, most (67%) of the 
participants were females. Their ages ranged between 
20 to 55 years; however, the majority was between 26 
and 30 years of age. Their teaching experience ranged 
from .5 to 16 years with a mean of 4.5 and a median of 
2.6 years. As expected, most (83%) of the participants 
were nurse educators, and 61% had a BSN degree, 
while 30% were prepared at the master’s level. 
Although all 12 non-nurse participants had a master’s 
degree, only 10 (13.9%) of the nurse participants had a 
master’s degree. There were no significant differences 
between the participants in the intervention and control 
groups on any of these variables. 

Contextual Factors 

As shown in Tables 2 and 3, the distribution of the 
class sizes and their physical environment were almost 
identical on the first and second observation; however, 
the mean time for class duration was 65 (SD = 24) and 
59 (SD = 21) minutes, respectively. To control the 
effect of the class time on the number of questions 
asked pre- and post-intervention, the number of 
questions variables were re-expressed by dividing 
questions on the duration. 

Intervention Outcome 

An ANOVA for the total number of questions 
showed no overall difference from the first to the 
second observation period (pre-post main effect F ul0 = 
.055, p > 0.05), and there was no overall difference 
between the intervention and control groups (between 
group main effect F ul0 = .005 ,p > 0.05). As depicted in 
Table 4, the mean for the total number of questions in 
the intervention group increased from the first (25.91) 
to second (32.45) observation, but decreased in the 
control group (33.17 to 24.43). However, this 



Gul et al. 


Promoting Critical Thinking 43 


Table 1 


Characteristics of Participants 



Variables 

n 

% 

Gender 

Female 

48 

66.7 


Male 

24 

33.3 

Age in years 

Up to 25 

5 

7.0 


26-30 

27 

38.0 


31-35 

17 

23.6 


36-40 

17 

23.9 


41-50 

6 

8.4 

Professional qualification 

Master’s 

22 

30.6 


BSN 

44 

61.1 


Diploma in nursing and midwifery with post-basic teaching diploma 

4 

5.6 


Diploma in nursing specialty diploma 

2 

2.8 

Faculty type 

Nursing 

60 

83.3 


Non nursing 

12 

16.7 

Teaching experience in 

1-1.5 

8 

11.1 

years 

2-4 

8 

11.1 


5-10 

32 

44.4 


11-16 

12 

16.9 


>16 

8 

11.2 

Formal training in CT 

Yes 

0 

0.0 


No 

72 

100.0 



Table 2 

Contextual Information of Observed Classes: Duration 



I s ' observation 

2 nd observation 


Intervention group 

Control group 

Intervention group 

Control group 

Variables 

M SD 

M SD 

M SD 

M SD 

Duration of the Class 

1:04 0:25 

1:05 

1:07 0:24 

0:56 0:20 

0:55 

1:02 0:23 

(Hour: minutes) 

Overall 


Note. Group n = 36 for all four observations. 


Table 3 


Contextual Information of the Observed Classes: Number of Students and 
_ Conduciveness of the Physical Environment _ 


Variables 

1 st observation 

2 nd observation 

Intervention group 
n (%) 

Control group 
n (%) 

Intervention group 
n (%) 

Control group 
n (%) 

Number of Students in the Class 




<20 

11 (30.6) 

9 (25.0) 

13 (36.1) 

12(33.4) 

21-30 

10 (27.8) 

12 (33.3) 

5 (13.9) 

10 (27.8) 

31-40 

7(19.4) 

8 (22.2) 

10 (27.8) 

5 (13.9) 

>40 

8 (22.2) 

7 (19.4) 

8 (22.2) 

9(25) 

Conduciveness of the Physical Environment 




Not at all 

2 (5.6) 

1 (2.8) 

3 (8.3) 

1 (2.8) 

To some extent 

14(38.9) 

19 (52.8) 

15 (41.7) 

16(44.4) 

To great extent 

20 (55.6) 

16 (44.4) 

18 (50) 

19(52.8) 


Note. Group n = 36 for all four observations. 
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Table 4 


Descriptive Statistics: Types and Levels of Questions (Per Hour) 


Variables 


1 st observation 
(pre-intervention) 

2 nd observation 
(post-intervention) 

Group 

n 

M 

SD 

n 

M 

SD 

Total number of 

Intervention 

36 

25.9 

29.5 

36 

32.5 

29.4 

questions 

Control 

36 

33.2 

33.2 

36 

24.4 

30.3 


Total 

72 

29.5 

31.4 

72 

28.4 

29.9 

Higher order questions 

Intervention 

36 

2.8 

2.9 

36 

6.0 

5.2 


Control 

36 

2.1 

2.8 

36 

2.0 

2.3 


Total 

72 

2.4 

2.9 

72 

4.0 

4.5 

Lower order questions 

Intervention 

36 

18.4 

23.4 

36 

19.8 

19.7 


Control 

36 

22.7 

19.2 

36 

16.5 

22.0 


Total 

72 

20.5 

21.3 

72 

18.2 

20.8 

Other types (e.g., 

Intervention 

36 

3.4 

4.7 

36 

4.9 

7.5 

facilitative: probing, 

Control 

36 

6.5 

9.7 

36 

4.3 

5.8 

clarifying, rhetoric) 

Total 

72 

4.9 

7.7 

72 

4.6 

6.7 

Vague questions 

Intervention 

36 

1.5 

2.2 

36 

1.4 

3.4 


Control 

36 

2.3 

6.5 

36 

1.5 

2.3 


Total 

72 

1.9 

4.8 

72 

1.4 

2.9 


interaction effect was not statistically significant 
(interaction effect F hl0 = 2.656, p = >0.05). 

In an ANOVA for the number of higher-order 
questions, the number increased from the first to the 
second observation period (pre-post main effect F ul0 = 
7.874 ,p = 0.006). However, that increase occurred only 
in the intervention group (interaction effect F\, 70 = 
8.265, p = 0.005), though this resulted in an overall 
difference between the intervention and control groups 
(between group main effect F ul0 = 15.173 ,p < 0.001). 

As depicted in Table 5, the means for the 
educators’ teaching strategies reflect a positive change 
from the first to second observation in both the 
groups. The ANOVA for the use of lecturing with a 
slide presentation indicated that the overall difference 
from the first to second observation was statistically 
significant (pre-post main effect F ul0 = 8.294, p = 
0.005). Although the overall difference between the 
intervention and control groups was not significant 
(between group main effect F ul0 = 0.674, p > .05), 
there was an interaction indicating that the increase 
from the first to the second observation period was 
statistically higher in the intervention group 
(interaction effect F\, 10 =5.308, = 0.042). Field notes 

supported that most teachers, who used a PowerPoint 
presentation, identified objectives for their class and 
were better organized to address the required content 
of their topic. However, those teachers who did not 
use a PowerPoint presentation usually began with the 
topic of the class and used personal notes to elaborate 
on the content relevant to the topic. Consequently, the 
teacher-student interaction was affected because the 
students had to concentrate more on listening and 


taking notes than on reflecting and internalizing the 
content. 

In the ANOVA for teachers’ use of active teaching 
strategies, the overall number of strategies increased 
from the first to the second observation period (pre-post 
main effect F\, 10 = 4.310,/? = .042). However, there 
was no overall difference between the intervention and 
control groups (between group main effect F ul0 = .464, 
p > 0.05). Likewise, the use of active teaching 
strategies by the intervention group in the second 
observation was not significantly higher than the 
control group (interaction effect F hl0 =.\12,p > 0.05). 

The mean scores with standard deviations and 
frequencies for each item on the educators’ facilitation 
skills have been provided in Table 6. Accordingly, the 
mean score for teacher-student interaction in the 
intervention group increased slightly from the first to 
the second observation as compared to that of the 
control group. Similarly, the mean for the teachers’ 
attitude for mutual respect increased from the first to 
the second observation. However, these differences 
were not statistically significant. Moreover, the mean 
for the teachers’ response to the students’ needs or 
concerns did not change from the first to the second 
observation. 

Unlike the first three items, an ANOVA for 
teachers’ encouragement to students for asking 
questions showed a significant interaction (interaction 
effect F h66 = 4.554, p = .037) such that the increase 
from the first to the second observation period occurred 
only in the intervention group. 

The dictation of notes was significantly reduced in 
the intervention group at the second (post intervention) 
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Table 5 


Descriptive Statistics: Teaching Strategies __ 

1 st observation 2 nd observation 


Intervention 

Control 

Intervention 

Control 

group 

group 

group 

group 


Teaching strategies 

M (SD) 

M (SD) 

M (SD) 

M (SD) 

Lecture with slides 

20 (55.6) 

21 (58.3) 

29 (80.6) 

22 (61.41) 

presentation 

0.56 (0.50) 

0.58 (0.50) 

0.81 (0.40) 

0.61 (0.49) 

Lecture without slides 

16(44.4) 

15 (41.7) 

7(19.4) 

14(38.9) 

presentation 

Use active teaching 

6(16.7) 

3(11.1) 

10(27.8) 

9 (25.0) 

strategies* 

0.17(0.37) 

0.14(0.42) 

0.33 (0.54) 

0.25 (0.45) 

Did not use active teaching 
strategies 

30 (83.3) 

33 (88.9) 

26 (72.2) 

27 (75.0) 

Note. Group n = 36 for all four observations. *Only one teacher (2.8) in the first observation (control group) and 
another teacher (intervention group) in the second observation used two active teaching strategies. 




Table 6 




Descriptive Statistics: Educators ’ Facilitation Skills 



1 st 

observation 

2 nd 

observation 


Intervention 

Control 

Intervention 

Control 

Variables 

group 

M (SD) 

group 

M (SD) 

group 

M (SD) 

group 

M (SD) 

Promote teacher-student 
interaction 

1.44 (0.56) 

1.50 (0.56) 

1.61 (0.49) 

1.47 (0.65) 

Demonstrates attitude of 
mutual respect 

1.69 (0.47) 

1.67 (0.53) 

1.72 (0.51) 

1.56(0.61) 

Responsive to students’ 
needs and concern 

1.41 (0.50) 

1.44 (0.56) 

1.41 (0.62) 

1.34 (0.65) 

Encourages students when 
they ask questions 

Instructor dictate notes 

1.44 (0.61) 

0.25 (0.55) 

1.53 (0.58) 

0.28 (0.62) 

1.61 (0.49) 

0.03 (0.17) 

1.31 (0.64) 

0.31 (0.62) 

Note. Group n = 36 for all four observations. 


observation. Similarly, an ANOVA yielded a 
significant interaction (interaction effect F\, 10 = 4.103, 
p = .047) for teachers’ behavior of dictating notes to 
their students. This behavior decreased from the first to 
the second observation period only in the intervention 
group. 

Discussion 

This study was conducted to determine whether 
teachers’ pedagogical skills for the promotion of 
students’ CT could be enhanced by providing them with 
formal training about the ontology and epistemology of 
CT. The results of this study are very encouraging as 
several positive changes were noted in the educators’ 
classroom teaching practices post intervention. Similar 
to the existing literature (Craig & Page, 1981; Hsu, 
2007; Ijaiya et ah, 2010; Phillips & Duke, 2001; 
Profetto-McGrath et ah, 2004; Sellappah et ah, 1998; 


Wink, 1993), pre-intervention findings in this study 
also indicate that on average the educators’ asked fewer 
high level questions than lower level questions. 
However, a statistically significant increase was noted 
in the educators’ ability to pose higher order cognitive 
questions after they had completed the intervention. 
These findings coupled with the results from some 
previous studies (e.g., Craig & Page, 1981; Wink, 
1993) affirm that educators need to improve their 
questioning skills. Formal training and coaching 
sessions can help them improve these skills. Though the 
level of questions has to be appropriate with the 
learners’ level of familiarity with the content (Phillips 
& Duke, 2001), it is the higher level questions that 
promote students’ CT (Redfield & Rousseau, 1981). 

In a recent study focusing on the impact of 
teaching practices on students’ CT skills, Shim and 
Walczak (2012) reported that in addition to asking 
challenging questions, well-organized presentations by 
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faculty were also associated with the students’ gain in 
CT. In our study, we observed a significant increase in 
the teachers’ use of a PowerPoint presentation by the 
intervention group during their second observation. 
Moreover, some increase, albeit not statistically 
significant, was observed in their use of active teaching 
strategies compared with that of the control group. The 
inability to reach a significant level of difference could 
have resulted from several factors. Many participants in 
our workshop had highlighted that their students and 
administration expected faculty to teach extensive 
amounts of content in a given time. Moreover, except 
for one school, 70% of the students’ assessments in the 
nursing degree programs in Pakistan are centralized by 
their respective universities. The centralized 
assessments usually consist of paper and pencil 
examinations. Therefore, teachers may be reluctant to 
increasingly rely on active teaching strategies, but may 
be more inclined to cover the content through 
interactive lectures. Faculty workload may well be 
another reason because in addition to the knowledge of 
active teaching strategies, faculty need time to plan, 
prepare and use such strategies (Shell, 2001). 
Concurrent with the recommendation of other 
researchers (Cassum et al., 2013; Mangena & Chabeli, 
2005; Zygmont & Schaeffer, 2006), a shared 
philosophy with coordinated efforts among faculty, 
students and administration would be required to 
change the entire culture of higher education in 
Pakistani universities. 

Shim and Walczak (2012) asserted that the 
development of CT in students requires the teachers to 
balance the cognitive challenges with support, which 
necessitates good facilitation skills. With regards to 
change in the educators’ facilitation skills after the 
intervention, we noted a desirable change in four items 
(Table 6). However, when compared with the control 
group, the change was statistically significant for two 
items: “encouragement to students for asking question” 
and “reduction in dictation of notes.” 

Most study participants were fairly young and had 
limited teaching experience (M = 4.4, SD = 4.07). It 
was disconcerting to learn that none of the participants 
had any formal preparation (course, seminar, or 
workshop) in CT before their participation in the 
current study. In their assessment of faculty CT in the 
USA, Zygmont and Schaefer (2006) also noted that 
most (78.4 %) of their participants had no education in 
CT while their average teaching experience was 14.47 
years. 

As noted in the standard deviation of items scores 
(Table 4), data for both groups and points of 
measurement indicate extensive variability in the 
teachers’ questioning skills, which is not a new 
phenomenon or surprising. What is important to note is 
that the desired skills, albeit at varying levels, can be 


enhanced in most teachers. Empirical evidence 
consistently suggests that learning skills develop faster 
if they are taught explicitly along with the relevant 
content (Weimer, 2002). In agreement with the 
recommendations of other scholars and researchers 
from different parts of the world (Behar-Horenstein & 
Niu, 2011; Choy & Cheah, 2009; Zygmont & Schaefer, 
2006), findings from this study affirm the need for a 
formal and structured training for teachers’ CT, so that 
they can develop, appreciate and apply teaching 
practices that are known to promote students’ critical 
thinking. 

Drawing on the work of renowned scholars (e.g., 
Broofield, Lipman), Behar-Horenstein and Niu (2011) 
maintained that to emulate CT in their teaching 
practices, teachers must be able to differentiate ordinary 
thinking from critical thinking, and they must be able to 
“understand process that constitute critical thinking” (p. 
27), and employ instructional strategies aimed at 
developing these processes. In addition to the 
knowledge and skills about CT, our intervention with 
the teachers in this study suggests that teachers must be 
given the opportunity to explore/externalize their own 
attitude about CT and address the myths that may 
preclude them from changing their practices. For 
instance, a number of participants in our interventional 
workshops identified several cultural and institutional 
barriers, such as limited resources (e.g., books, space, 
budget for teaching and learning material) and the 
expectations to just complete the content, which 
generally prevented them from using more active 
teaching strategies. However, with deeper reflections 
and discussion on those barriers and the demonstration 
of several active teaching strategies, the participants 
were able to realize and dispel such myths. Zygmont 
and Schaefer (2006) maintained that “the transition 
from being inclined to thinking critically and actually 
having the skill” (p. 260) requires a combination of 
time, experience and mentorship. 

Limitations 

This study has several limitations. Block 
randomization would have been a better option to 
prevent contamination of the participants in the control 
group, but block randomization was not implemented 
for two reasons. First, the faculty size in one school was 
four times larger than other schools. Second, it would 
have been very difficult for any school to allow all of 
their enrolled faculty members to attend the 
intervention workshops at the same time. To minimize 
the risk of contamination, participants in the 
intervention group were briefed about the study design, 
and they were asked not to share what they learned with 
colleagues assigned to the control group at their 
institution. Although the possibility of the Hawthorne 
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effect during the second observation could not be ruled 
out as the participants were not blinded to their 
assignment, they were not aware of the assessment 
parameters. Another limitation was that data from 
various cities were collected by different members of 
the research team or by a local data collector. However, 
to control the variations among data collectors, 
members of the team were involved in a detailed 
discussion about the data collection process before and 
after the pilot testing, and the data collectors were 
properly trained for the same process. Moreover, all the 
recorded questions were transcribed and coded by the 
research assistant; the coding was verified by a member 
of the team. Although Bloom’s (1956) taxonomy for six 
levels is well established, in our experience, the 
differentiation between the last three levels (i.e., 
analysis, synthesis, and evaluation) was found 
challenging, especially when we had hundreds of 
questions to evalutate. To overcome this issue, we 
analyzed the questions for high and low order, which 
was appropriate for our study question. 

Conclusion 

This study adds to the knowledge of faculty 
development to foster CT in their teaching discourses. 
Previous research on this topic was more focused on 
the teachers’ ability to ask higher order questions after 
an educational intervention. This study offers an 
example of extending training efforts beyond the 
teachers’ questioning skills and the need for exploring 
contextual factors that may be inhibiting students’ 
thinking. Moreover, training endeavors to enhance CT 
could be more beneficial when a multidisciplinary 
approach is employed. Although further research is 
needed, this study affirms the plea that some formal 
training is necessary to enhance educators’ CT skills if 
they are expected to enhance CT in their teaching 
practices. 
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Appendix-A 

Intervention Workshops on Critical Thinking 


Topic/Content 

Strategies 

• Definitions and descriptions of CT 

• Multidimensionality of CT as an attitude, skill, art, 
outcome, process, reflection 

• Characteristics of a critical thinker: Knowledge, skills, and 
attitudes/disposition 

• Clarifying of perceptions, and accepting diverse opinions, 
avoiding stereotypes 

• CT/problem-solving/decision-making/creative thinking 

• PowerPoint interactive presentation 

• Reflections on pictures & images to explore varied 
perceptions, thoughts, and ideas about concepts related to 
thinking and CT. 

• Questioning 

• Debate on CT skills vs. CT dispositions 

• Small group exercise (e.g., fish bowl) 

• Large group discussion 

• Importance of critical thinking in nursing practice and 
education 

• Challenges and barriers in teaching CT 

• Difference between active and passive strategies 

• Learning environment and student engagement 

• Teaching strategies to promote critical thinking in students 

• Role play depicting traditional classroom where teacher 
does one way teaching and overload content on slides, 
passive learning, poor questioning followed by reflection 
on the role play 

• Reflection: Think of past and recall your favorite teacher 

• Small groups activity for development of concept map on 
barriers to CT 

• Development of Pros/Cons Grid by Think pair & share 
activity 

• Jigsaw for active teaching strategies 

• Importance of questioning in nursing 

• Bloom’s taxonomy 

• Types, Quality, and levels of questions 

• Research findings on questioning skills amongst educators 

• Instructional objectives for each level of blooms taxonomy 

• Writing instructional objectives; Cognitive domains ladder 
(6 levels) 

• Interactive discussion using PowerPoint presentation; 
Questioning 

• Muddiest point; clarification of major points related to 
Blooms taxonomy 

• Develop questions in small groups, present and critique in 
large group. 

• Modify the given questions (closed to open, low to high 
order) 

• Think pair & share; Activity on identifying correct or 
incorrect objectives and modifying the latter 

• Importance and Purpose of a course grid and its 
components 

• The concept of alignment in class plan, course, and 
curriculum 

• Steps and skills for identifying coherence in various 
component of a course 

• Curricular alignment Triangle framework (Article by Lorin 
Anderson) 

• Take home assignment briefing and expectations 

• Participants experience of the workshop 

• Interactive discussion using PowerPoint presentation; 

• Planned and unplanned questions 

• Application of alignment exercise in small group activity- 
participants required to develop a class plan including 
objectives, content and teaching strategies. 

• Critique of class plan in large group for alignment 

• CT Survey questionnaire; workshop evaluation 

The learners returned after 14 weeks for the second workshop 

• Share experiences and challenges of completing assigned 
task 

• Sharing of personal experience in application of knowledge 
and skills in learned in the first workshop. 

• Barriers to application and ways to overcome the identified 
barriers 

• Facilitators of CT Reflections and reflective 

• Evaluate selective course grid for alignment 

• Conclusion & Workshop Evaluation 

• Interactive discussion using PowerPoint presentation 

• Sharing synopsis of the experience 

• Planned and unplanned questions 

• Synthesis of literature on Reflection and presentation of 
synthesis in a concept map 

• Discussion on the presentation of concept maps. 

• Reflections on the quality of discussion 

• Small group exercise to critique for alignment and 
presentation of the finding 
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Appendix B 

Checklist for Classroom Observations 


Code: 


Topic of the Class:_ 

Code of the school:_; City:_ 

Duration of the class: Start time:_End time:_ 

Break: □ Yes □ No ; Total class duration excluding break time:_ 

Observed class is of which programme: 4-year BSN □ Post-RN BSN □ 


Note: circle the appropriate response and write comments in the field notes as appropriate. 


1. 

What was number of students in the class? 

1. <20 

2. 21-30 

3. 31-40 

4. >50 

2. 

Was the Physical Environment of the class conducive to learning? 

1. Not at all 


Descriptors: seating arrangement, comfortable seating, adequate light and 
ventilation, free of distractions (noise) 

2. To some extent 

3. To great extent 

3. 

Did the instructor promote teacher-student interaction? 

1. Not at all 


Descriptors: verbal and non-verbal: level of distance between teacher and 

2. To some extent 


students, attentive (eye contact, listening, nodding) when students are 
talking, approachable, non-threating, but welcoming approach to students, 
invites questions or participations. 

3. To great extent 

4. 

Did the instructor demonstrate an attitude of mutual respect towards 
the students? 

1. Not at all 

2. To some extent 


Descriptors: Verbal (language and tone of communication) and non¬ 
verbal gestures reflecting respect, open mindedness-acceptance of different 
views. Students appears to feel safe to express their feelings, as their ideas 
and opinions are valued 

3. To great extent 

5. 

Was the instructor responsive to students’ needs and concerns? 

1. Not at all 


Descriptors: Teacher responds to student’s questions; provides 
clarification, explanation as needed; demonstrates flexibility- make changes 
on the students’ request, e.g. need for a break. 

2. To some extent 

3. To great extent 

4. Not applicable 

6. 

Did the instructor encourage students if they asked questions? 

1. Not at all 


Descriptors: Compliment students for asking questions e.g. good question! 
Thanks for asking/raising this issue. When the question is not clear, doesn’t 
ridicule, but probe to understand the question. 

2. To some extent 

3. To great extent 

4. Not Applicable 

7. 

Did the instructor dictate notes to the students? 

1. Not at all 


Descriptors: Teacher’s emphasizes on noting down of content, e.g. 
copy/note what is on the board/slide; write it... 

2. To some extent 

3. To great extent 

9. 

Did the instructor use any teaching aids/resources? 



Descriptors: 

Board (white/black), Models, Charts, video etc. 

1. Multi Media (PP-slides) 

2. Others: 
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10. 

Did the instructor use any active teaching strategies? (circle all that are 



appropriate) 

1. Group work 



2. Role Play 


Descriptors: Teachers direct students for an activity, explains the context 

3. Debate 


and process, make the student reflect on what they did and learn. 

4. Concept mapping 



5. Others 


Field Notes: (any observation that may facilitate or inhibit students thinking) 


Recorder name and signature: 


Date: 
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Appendix C 

The Types and Levels of Questions 


Types of 
Questions 

Levels of Questions: Required Cognitive Activity 

Examples 

Lower order 

Knowledge: the lowest level of cognitive thinking that entails 
remembering or recalling factual information, it includes 
memorization of definitions, formulae or procedures. 

Which organ in the body produces 
insulin? 

What is peritonitis? 

Comprehension: understanding of information, usually 
restating the information with some reorganizing, but without 
relating it to other concepts. 

How conduction system of the heart 
works? 

What is done to a patient blood in 
plasmapheresis? 

Application: Problem solving or application of learned 
material in new situations with minimal prompting of the 
appropriate rules, principles, or concepts. 

What are some possible Nursing 
diagnoses for patient with Acute 
renal failure? 

Higher order 

Analysis: Breaking an idea into its component or parts for 
logical analysis or reasoning to support a conclusion. 

How would you confirm that 
whether it is respiratory or metabolic 
acidosis? 

Synthesis: Combining ideas into a statement, plan, product, 
etc. 

What is the role of diet and exercise 
in health? 

Evaluation: Evaluating or making a judgment about 
something using some criteria or standard. 

Which is the most appropriate 
nursing management for an elderly 
patient having stroke? 

Vague 

Questions that are difficult to interpret, because the given 
information is incomplete or asked in an illogical order. 

What is the intake of a normal 
person? What do you think about 
personal development? 




